Exploring the Performance of Boolean Retrieval Strategies for Open Domain Question Answering

This paper is concerned with systematically exploring and evaluating a range of possible boolean retrieval strategies for use within a Question Answering (QA) system. We firstly set out two evaluation metrics — coverage and recall — which are specifically designed for use in evaluating retrieval performance in a QA context, and apply these measure in quantifying the performance of some standard ranked retrieval systems for this purpose. We then consider a series of possible boolean retrieval strategies for use in QA, which concern the way that boolean queries are generated from questions to retrieve passages relevant to finding the question’s answer, and evaluate their performance. This line of research should ultimately lead to an increased understanding of how best to formulate retrieval strategies for QA and of which component methods can usefully contribute to such strategies.